Method and apparatus for colposcopic examination

ABSTRACT

The colposcopic examination apparatus comprising an optical head with a lens capable of changing focusing distance and an image acquiring unit capable of registering images created by the lens, as well as a computer system designed to control the lens and the image acquiring unit, to obtain a series of images created by the lens at different focusing distances and to perform focus stacking of the obtained images, wherein the computer system is capable of automatic recognition of the chosen examined object on the image created by the lens and of further automatic setting of shooting parameters, focusing distance of the lens and number of images to be created by the lens at different focus distances for the computer system to perform focus stacking and to generate the output image of the examined object with extended depth of field. Also disclosed is a method of colposcopic examination.

FIELD OF THE INVENTION

The invention relates to medicine, specifically to gynecology, inparticular to the method and the apparatus for colposcopic examinationinvolving processing of digital images.

BACKGROUND OF THE INVENTION

Colposcopic examination is based on revealing the difference in mucosalrelief and vessel appearance in normal and pathological epithelium ofthe vaginal portion of cervix and is an efficient method to detectlesions of ectocervical and endocervical epithelium, to assess theirnature and localization, to perform differential diagnosis of benignlesions of ectocervix, vagina and vulva, to assess feasibility ofcervical biopsy, to determine the area and the method of histologicalexamination sampling, to choose the treatment method of the identifiedpathology.

Obtaining colposcopic images presents difficulties that have not beensolved by the prior art. Normally the apparatus for colposcopicexamination is positioned at a small distance from the chosen object(15-30 cm). Such range allows to obtain high optical resolution,sufficient illumination of the object, at the same time ensuring accessfor the doctor's manipulations, such as applying solutions, tissuesampling etc. At such a range the depth of field, which is the zonewhere examined objects appear acceptably sharp, constitutes from severalmillimeters for small matrix cameras (⅓ inch) to tenths of a millimeterfor full frame cameras. Quality and resolution are normally proportionalto the matrix size, i.e. the depth of field decreases as the imagequality requirements grow.

At the same time, shallow depth of field is a negative factor when usingimages obtained during of colposcopic examination, since just a smallpart of the examined object is acceptably sharp on each of the images.Several frames to several dozens of frames are needed to obtaininformation about the entire object. Large number of frames complicatestheir further analysis and increases probability of some of the examinedobject areas being captured in quality too low for further analysis.

One of the prior art solutions of the shallow depth of field issue isthe focus stacking method, a tool allowing to extend the depth of fieldof a digital image. This method is based on merging multiple sourceimages of the same scene, usually captured from the same point of viewwith different depth of field (for instance, a series of static imagescaptured from a fixed point of view or a series of frames from astationary video camera), to produce an output image with a depth offield larger than on any of the source images.

In order to render the entire examined object sharp through focusstacking, source images should meet certain requirements. In particular,the areas of depth of field of subsequent source images should haveoverlapping areas in order to provide sufficient data. Besides, somefocus stacking methods require that the depth of field area of theseries of output images does not change the shift direction (i.e. thefocus has to be shifted in the same direction—either towards infinity orcamera).

Capturing images for focus stacking is called focus bracketing. Normallyeach frame in focus bracketing is captured with the same settings as theprevious one, but the focus point of the lens, and therefore, the depthof field, is being shifted by a specified step in a specified direction.Most often focus bracketing is done according to the followingprocedure: the first frame is taken with focus at the nearest point ofthe object, the following ones are captured with the shift of focustowards infinity until the farthest point of the object is in focus.

Performing manual focus bracketing of images captured during colposcopicexamination is problematic. The major problem is to determine thenearest and the farthest points of the examined object. Mucosa surfacesare normally characterized by low contrast level and it is quitedifficult for the specialist conducting examination to define if therelevant area is in focus or not. In addition, even examination of sucha simple three-dimensional figure as a raised hemisphere of the cervixsurface present difficulties for determining the nearest point of theexamined object, due to the cervix often being inclined, deformed andnon-uniformly illuminated. Colposcopic examination of other objectsinvolves even more difficulties due to their three-dimensional shape.

SUMMARY OF THE INVENTION

According to the present invention, the problem of the prior art issolved by developing a method and an apparatus for colposcopicexamination.

In the first aspect the claimed invention is an apparatus comprisingoptical head with a lens designed with possibility to modify focusingdistance, and an image acquiring unit capable of registering imagescreated by the lens, as well as a computer system designed to controlthe lens and the image acquiring unit, to obtain a series of imagescreated by the lens at different focusing distances and to perform focusstacking of the obtained images, wherein the computer system is capableof automatic recognition of the chosen examined object on the imagecreated by the lens and of further automatic setting of shootingparameters, focusing distance of the lens and number of images to becreated by the lens at different focus distances for the computer systemto perform focus stacking and to generate the output image of theexamined object with extended depth of field.

In the second aspect, the claimed invention is the method of conductingcolposcopic examination comprising selection of the examined object,obtaining series of images created by the lens at different focusingdistances by a computer system of the apparatus for colposcopicexamination, and performing focus stacking of the created images. Oncethe examined object has been selected it's automatically recognized onthe image created by the lens, with further automatic setting ofshooting parameters, focusing distance of the lens and number of imagescreated by the lens at different focusing distances required for thecomputer system to perform focus stacking and to generate the outputimage of the examined object with extended depth of field.

The Summary of the Invention is provided to introduce the main conceptof the invention in a simplified form that is further described below inthe Detailed Description of the Invention. The Summary of the Inventionis not intended to identify essential features of the claimed subjectmatter, nor is it intended to be used to limit the scope of the claimedsubject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an example embodiment of the method ofcolposcopic examination according to one of the preferred embodiments ofthe present invention.

FIG. 2 is an example of a source image created by the lens with certainfocusing distance in its first position, where a part of gynecologicalspeculum is in the area of depth of field.

FIG. 3 is an example of a source image created by the lens at differentfocusing distances in its first position, where a portion of cervix isin the area of depth of field.

FIG. 4 is an example of a source image created by the lens with certainfocusing distance in its second position, where a part of gynecologicalspeculum is in the area of depth of field.

FIG. 5 is an example of a source image created by the lens with certainfocusing distance in its second position, where the portion of cervixclosest to the lens is in the area of depth of field.

DETAILED DESCRIPTION OF THE INVENTION

There are different apparatus for colposcopic examination producingcolposcopic images of the examined object that require further digitalprocessing.

Pursuant to the present invention, the apparatus comprises an opticalhead, a lens with possibility of changing focusing distance, and animage acquiring unit capable of registering images created by the lens,as well as a computer system capable of controlling the lens and theimage acquiring unit, acquiring series of images created by the lens atdifferent focusing distances and performing focus stacking of producedimages, wherein the computer system is capable to automaticallyrecognize the preselected examined object on the image created by thelens, with further automatic setting of shooting parameters, focusingdistance of the lens and number of images to be created by the lens atdifferent focusing distances, required for the computer system toperform focus stacking and to produce the output image of the examinedobject with extended depth of field.

According to the preferred embodiment of the invention, the computersystem comprises at least one processor and at least one computerreadable medium communicatively connected with at least one processor,as well as program instructions stored on at least one computer readablemedium and executable by at least one processor.

The program instructions described herein are executable by at least oneor several hardware logic components. Examples of possible types ofhardware logic components that can be used (without limitation) includefield-programmable gate arrays (FPGAs), application-specific integratedcircuits (ASICs), systems-on-a-chip (SOCs), complex programmable logicdevices (CPLDs), etc. Program instructions for carrying out operationsof the present invention may be represented by assembler codeinstructions, instruction-set-architecture (ISA) instructions, machineinstructions, machine-dependent instructions, set of micro commands,base software instructions, state data, or another source code or objectcode written in any combination of one or more programming languages,including object oriented programming languages such as Smalltalk, C++or the like, and conventional procedural programming languages such as Cor C-like programming languages.

The program instructions may be executed entirely on the user's computerbeing a part of the computer system, partly on the user's computer beinga part of the computer system as a stand-alone software, partly on theuser's computer being a part of the computer system and partly on aremote computer or entirely on a remote computer or server. In thelatter scenario, the remote computer may be connected to the user'scomputer through any type of network, including local area network (LAN)and wide area network (WAN), or it can be connected to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

In some embodiments, for example, in electronic circuitry, programmablelogic devices, field-programmable gate arrays (FPGA), or programmablelogic arrays (PLA) can execute computer readable program instructionsoptimizing the information about the state of such instructions foradjustment of electronic circuitry in order to implement the aspects ofthe present invention.

The computer readable medium includes volatile and non-volatile,removable and non-removable media applicable in any method or technologyof storing such information as computer readable instructions, datastructures, program modules, etc.

Computer readable medium may include random access memory (RAM),read-only memory (ROM), electrically-erasable programmable ROM (EEPROM),flash memory or another memory technology, CD-ROM, DVD format compactdisc or another optical storage, magnetic cassettes, magnetic tape,magnetic disk storage or other magnetic storage device, as well as anyother non-transmission medium that can be used to store information foraccess by a computing apparatus. Said computer readable storage mediumis a non-transitory one.

According to the preferred embodiment of the present invention, thecomputer system is capable of automatic setting of shooting parameterschosen from a group that in the preferred embodiment of this inventionincludes focusing distance value, aperture, sensitivity, exposure timevalues, white balance settings, brightness level of flash or additionalillumination, though this settings list is not exhaustive.

According to the preferred embodiment of the present invention, thecomputer system has possibility to automatically recognize the examinedobject through at least two iterations, where the lens creates imagesfor preliminary recognition of the examined object and furtherrefinement of obtained data about the examined object. The computersystem automatically sets shooting parameters for at least one iterationduring automatic recognition of the examined object.

Also according to the preferred embodiment of the present invention, thecomputer system is capable of using data selected from the groupincluding examined object data, data on additional gynecologicalequipment parameters, patient data, as well as capable of acquisitionand storage of the patient's colposcopic examination results data.

Pursuant to another preferred embodiment of the present invention, thecomputer system can generate a three-dimensional model of the examinedobject based on the obtained images.

Also according to the preferred embodiment of the present invention, thecomputer system can position the optical head relative to the selectedexamined object to ensure optimal shooting conditions for the examinedobject.

Pursuant to yet another preferred embodiment of the present invention,the computer system can combine the current output image of the examinedobject with extended depth of field with previously obtained outputimage of the examined object with extended depth of field to detectchanges in the examined object over time.

According to the preferred embodiment of the present invention, theclaimed apparatus is designed to conduct colposcopic examination ofobjects selected from a group that includes areas adjacent to vaginalopening, vaginal opening, vagina walls, ectocervix and endocervix forfurther detection and classification of skin and mucosa lesions.

The preferred method of automatic recognition of the examined object isas follows. In cases when the examined object requires using thegynecological speculum as additional gynecological equipment mentionedabove, preliminary recognition is performed through matching of sampleimages that may be used by the computer system with the current sourceimage. So the data about the type and the size of currently usedgynecological speculum is obtained through interaction of the user, forinstance, with the user interface of the computer system of thecolposcopic examination apparatus used for implementation of the claimedmethod, and selection by the user of the required parameters from thelist of available parameters characterizing additional gynecologicalequipment. The user should have possibility to choose the type and thesize of currently used gynecological speculum in the settings. With allthe available information about the parameters of gynecological speculumthe computer system of the claimed apparatus with higher accuracy willperform automatic recognition of the gynecological speculum on the imagecreated by the lens and will determine the exact distance to thegynecological speculum, the angle to the gynecological speculum axis andthe approximate distance to the cervix being the examined object. Allthis information can be used for further setting of optimal shootingparameters, lens focusing distance, number of images etc. In particular,the computer system with the positioning unit ensures positioning of theoptical head by generating a command for its movement (lens movement) sothat the distance and the direction are optimal for the current examinedobject. Besides, the lens focusing distance shift step, the aperturevalue, the sensitivity depend on illumination level. For instance, thevaginal depth, the material and design of gynecological speculumsignificantly affect illumination level when photographing cervix. Athigh illumination level, it's optimal to use small aperture values(f11-f13) and larger lens focusing distance shift step value (focusbracketing step). At low illumination level, it's better to use highaperture value (f5.6) and smaller lens focusing distance shift step.After preliminary recognition of the gynecological speculum anddetermining its position, one or several additional iterations may berequired to define the position of the examined object more precisely.For instance, after correct positioning of the lens through movement ofthe optical head and setting of new shooting parameters, it's needed toprecisely determine the nearest and farthest point of the examinedobject. For cervix, this task is more complicated due to the mucosalsurfaces having very little detail. In this case the apparatus may relyon recognition of cervical canal and the preset information about thephysiological cervix shape instead of recognition of contrast details.

Unlike conventional pattern recognition such as detecting faces on theimage, implementation of the present invention may require performing atleast two iterations for preliminary recognition of the object using allthe available information, for instance, gynecological speculumparameters, physiological shape of cervix and other parts. This isespecially vital if the examined object does not require usingadditional gynecological equipment. The information used may alsoinclude patient data (height, weight, race, age), as well as data on thepatient's previous colposcopic examinations collected and stored by thecomputer system of the apparatus. Preferably, matching of sample imagesthat can be used by the computer system with the current image createdby the lens should be performed during preliminary recognition.

The three-dimensional model of the examined object obtained afterautomatic recognition and the information about the position of theapparatus optical head and the lens itself relative to the examinedobject allows to ensure further positioning of the optical head relativeto the chosen examined object in order to provide optimal shootingconditions for the examined object. Positioning can be ensured throughinstallation of the positioning unit on the colposcopic examinationapparatus allowing for automatic change of the optical head positionafter the computer system receives the corresponding command from theuser entered through computer interface, or by providing the requiredinformation to the user through user interface of the computer systemwith further manual positioning of the optical head.

The present invention is also described in the present patent claimthrough the colposcopic examination method implemented with the help ofthe above described apparatus.

The aspects of the present invention are described herein with referenceto the drawings.

FIG. 1 depicts a block diagram of an example implementation of thecolposcopic examination method 100 according to one of the preferredembodiments of the present invention. Said method 100 comprises examplesof specific steps of colposcopic examination. Some steps of the method100 or the entire method 100 (or any other processes described herein,or variations and/or combinations thereof) can be implemented as programinstructions (for instance, program code, one or several computerprograms or one or several applications) executed collectively by one orseveral processors, computer system hardware or the combination thereof.Program instructions can be stored on one or more computer readablemedia, for instance, as a computer program containing multipleinstructions executed by one or more processors. Computer readableinformation storage medium is a non-transitory one.

Block diagram of the method 100 begins with the block 102 illustratingthe step of selecting the object to be examined. The examined object canbe selected by the computer system of the colposcopic examinationapparatus receiving data about the object chosen by the user from thelist of available objects, for instance, through interaction of theuser, for instance, with the user interface of the computer system ofthe colposcopic examination apparatus implementing the said method 100.

The next block diagram 104 depicts automatic recognition of thepreliminary selected examined object on the image created by thecolposcopic examination apparatus lens. As described above, thepreferred method of automatic recognition of the examined object mayconsist in the following. In cases when the examined object requiresusing gynecological speculum as above mentioned additional gynecologicalequipment, the preliminary recognition is performed by matching sampleimages that can be used by the computer system, with the current sourceimage created by the lens, wherein the above mentioned sample images canbe uploaded to the computer system by the manufacturer or user and bestored there. Data about the type and the size of currently usedgynecological speculum is obtained through interaction of the user with,for instance, the user interface of the computer system of thecolposcopic examination apparatus implementing the claimed method 100and user selection of required parameters from the list of parameterscharacterizing available additional gynecological equipment. Thegynecological speculum is automatically recognized on the image createdby the lens using available information about the parameters ofgynecological speculum, and the precise distance to the gynecologicalspeculum, the angle relative to the gynecological speculum axis and theapproximate distance to cervix being, for instance, the examined object,are determined. Automatic recognition of the examined object can also beperformed without using additional gynecological equipment, in this casea larger number of iterations will be needed for preliminary recognitionof the examined object and further refinement of the obtained data aboutthe examined object.

The next is the block 106 depicting automatic setting of shootingparameters, focusing distance of the lens and the number of imagescreated by the lens at different focusing distances. Once the operationsof the block 104 have been performed, those of the block 106 should beperformed on the basis of the obtained data for refinement of theexamined object position or for creation of a series of images of thealready recognized examined object for further digital processing,executing either the full range of described operations or setting justone of the above stated parameters depending on whether this operationis performed after full automatic recognition of the examined object orafter one of iterations during automatic object recognition.

The next block 108 depicts the process of the colposcopic examinationapparatus computer system acquiring a series of images created by thelens at different focusing distances and performing focus stacking ofobtained images to produce the output image of the examined object withextended depth of field, wherein focus stacking can be performed in anyof the known methods, for instance, by the method based on the depth mapgeneration.

FIG. 2 depicts an image created by the lens in its first position whenfocusing at the examined object located 30 cm from the lens, withminimal aperture value, wherein the area of depth of field contains apart of gynecological speculum 200 with the parameters known to theuser.

FIG. 3 depicts an image created by the lens in its first position, butwith the focusing distance changed based on the data obtained by thecomputer system of the apparatus after processing of the image on theFIG. 2, wherein the portion of cervix 300, in particular the cervicalcanal, is in the area of depth of field.

Based on the analysis results of position of cervical canal andgynecological speculum from the FIG. 3 and FIG. 2 respectively, that isperformed by the computer system of the claimed apparatus in course ofautomatic recognition of the examined object, a reasonably preciseassumption about the position of the optical head and, respectively, thelens of the apparatus relative to the gynecological speculum axis ismade. In order to photograph all portions of cervix, the lens should bepositioned as close to the gynecological speculum axis as possible.Based on the information described above the position of the opticalhead and, respectively, the lens relative to the examined object ischanged automatically, if allowed for by the apparatus design, ormanually, and the above described operations are repeated in the newposition.

FIG. 4 depicts an image created by the lens with certain focusingdistance in its second position with a part of gynecological speculum400 in the area of depth of field, FIG. 5 depicts an image created bythe lens with certain focusing distance in its second position with theportion of cervix 500 closest to the lens in the area of depth of field.These images demonstrate that this lens position is optimal, the cervixis not hidden by the edges of gynecological speculum and the portion ofcervix closest to the apparatus is in focus. So further shooting ofcervix is done in this position for the lens to shoot a series of imageswith different focusing distances required for the computer system toproduce an image of the examined object with extended depth of field.

Although the examples of performing colposcopic examinations weredescribed in a language specific to the structural features and/ormethodological steps, it should be understood that the method ofconducting colposcopic examination, as well as the colposcopicexamination apparatus as defined in the appended claims are notnecessarily limited to the specific features or steps described above.Rather, the specific features and steps described above are disclosed asexamples of implementing the claims, and other equivalent features andsteps can be encompassed by the claims of the present invention.

1. Apparatus for colposcopic examination, comprising optical headcomprising a lens capable of changing focusing distance and an imageacquiring unit capable of registering images created by the lens, aswell as a computer system capable of controlling the lens and the imageacquiring unit, of producing series of images created by the lens atdifferent focusing distances and performing focus stacking of producedimages, wherein the computer system is capable of automatic recognitionof the preliminary selected object on the image created by the lens, andfurther automatic setting of shooting parameters, lens focusing distanceand number of images to be created by the lens at different focusingdistances, required to produce an output image with extended depth offield through performing of focus stacking by the computer system. 2.The apparatus of claim 1, characterized in that the computer systemcomprises at least one processor and at least one computer readablemedium communicatively connected to at least one processor, as well asprogram instructions stored on at least one computer readable medium andexecutable by at least one processor.
 3. The apparatus of claim 1,characterized in that the computer system is capable of automaticsetting of shooting parameters selected from the group includingfocusing distance value, aperture, sensitivity, exposure time values,white balance settings, brightness level of flash or additionalillumination.
 4. The apparatus of claim 1, characterized in that thecomputer system is capable of automatic recognition of the examinedobject through performing of at least two iterations for preliminaryrecognition of the examined object and further refinement of obtaineddata about the examined object.
 5. The apparatus of claim 4,characterized in that the computer system is capable of automaticsetting of shooting parameters for at least one iteration during ofautomatic recognition of the examined object.
 6. The apparatus of claim1, characterized in that the computer system is capable of using dataselected from the group including data about the examined object, aboutthe parameters of additional gynecological equipment, the patient data.7. The apparatus of claim 1, characterized in that the computer systemis capable of collection and storage of data about the results of thepatient's colposcopic examination.
 8. The apparatus of claim 1,characterized in that the computer system is capable of creating athree-dimensional model of the examined object based on produced images.9. The apparatus of claim 1, characterized in that the computer systemis capable of positioning the optical head relative to the chosenexamined object to ensure optimal shooting conditions of the examinedobject.
 10. The apparatus of claim 1, characterized in that the computersystem is capable of combining the current output image of the examinedobject with extended depth of field with a previously produced outputimage of the examined object with extended depth of field to detectchanges of the examined object over time.
 11. The apparatus of claim 1,characterized in that the apparatus is capable of performing colposcopicexamination of objects selected from the group including areas adjacentto vaginal opening, vaginal opening, vagina walls, ectocervix andendocervix for further detection and classification of skin and mucosalesions.
 12. Method of performing colposcopic examination, comprisingselection of the object to be examined, acquisition by the colposcopicexamination apparatus computer system of a series of images created bythe lens at different focusing distances, and performing focus stackingof acquired images, wherein after selecting the examined object, theautomatic recognition of preliminary selected examined object on theimage created by the lens is performed, as well as further automaticsetting of shooting parameters, focusing distance of the lens and numberof images to be created by the lens at different focusing distances,required to produce the output image of the examined object withextended depth of field through performing focus stacking by thecomputer system.
 13. The method of claim 12, characterized in automaticsetting of shooting parameters selected from the group includingfocusing distance value, aperture, sensitivity, exposure time values,white balance settings, brightness level of flash or additionalillumination.
 14. The method of claim 12, characterized in that theautomatic recognition of the examined object is performed through atleast two iterations for preliminary recognition of the examined objectand further refinement of obtained data about the examined object. 15.The method of claim 14, characterized in that the automatic setting ofshooting parameters is performed for at least one iteration duringautomatic recognition of the examined object.
 16. The method of claim12, characterized in that it comprises entry of the data selected fromthe group including data about the examined object, parameters ofadditional gynecological equipment, patient data.
 17. The method ofclaim 12, characterized in performing collection and storage of thepatient's colposcopic examination results data.
 18. The method of claim12, characterized in building of three-dimensional model of the examinedobject based on the obtained images.
 19. The method of claim 12,characterized in positioning of the optical head of the colposcopicexamination apparatus relative to the selected examined object in orderto ensure optimal shooting conditions for the examined object.
 20. Themethod of claim 12, characterized in combining the current output imageof the examined object with extended depth of field with a previouslyproduced output image of the examined object with extended depth offield to detect changes of the examined object over time.
 21. The methodof claim 12, characterized in conducting colposcopic examination ofobjects selected from a group that includes areas adjacent to vaginalopening, vaginal opening, vagina walls, ectocervix and endocervix forfurther detection and classification of skin and mucosa lesions.